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1.0  EXECUTIVE  SUMMARY 


1.  A  human  factors  study  was  conducted  on  the  B-1B  Alternative  Launch  Acceptability 
Region  (ALT  LAR)  displays.  The  study  objective  was  to  perform  a  human  factors 
evaluation  of  B-1B  ALT  LAR  displays  and  make  recommendations  to  B-1B  System 
Program  Office  Engineering  (SPO)  (ASC/YDE). 

2.  This  objective  was  studied  in  the  B-1B  Engineering  Research  Simulator  (ERS)  in  a 
simulated  full  mission  environment  using  five  B-1B  crews.  After  training,  the  crews 
flew  two  missions  with  Weapons  Systems  Operators  (WSO)  changing  positions 
between  missions.  Workload  was  measured,  a  questionnaire  was  given,  specific 
mission  data  were  collected,  and  all  missions  were  video  taped. 

3.  The  main  findings  of  the  study  are  that: 

a.  The  ALT  LAR  displays  allowed  the  Offensive  Systems  Operator  (OSO)  to 
average  92.4  percent  Bombs  on  Target. 

b.  All  of  the  WSO  ALT  LAR  modifications  received  a  “Very  Acceptable”  rating. 

c.  WSO  Target  Situational  Awareness  received  a  “Very  Acceptable”  rating. 

d.  Proposed  changes  to  the  primary  navigation  (Nav  Prime)  displays  on  the  OSO’s 
Multi-Function  Displays  (MFD)  were  rated  “Totally  Acceptable.”  These  changes 
were  independent  of  the  proposed  ALT  LAR  changes. 

e.  The  Map  Enable  function  increased  OSO  situational  awareness. 

f.  Track  Handle  control  of  the  OSO’s  MFD  was  rated  “Very  Useful,”  however, 
further  improvements  to  the  ERS  implementation  are  warranted. 

g.  Implementation  of  OSO  controlled  steering  commands  to  the  pilot’s  station  was 
insufficient  to  improve  pilot  situational  awareness. 
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2.0  INTRODUCTION 


Over  the  last  several  years  the  B-1B  System  Program  Office  Engineering  Department 
(ASC/YDE)  in  conjunction  with  the  Crew  Station  Evaluation  Facility  (CSEF)  of  the 
Engineering  Directorate  of  Aeronautical  Systems  Center  (ASC/ENFC)  has  conducted  a 
series  of  applied  human  factors  engineering  studies.  They  have  used  the  B-1B 
Engineering  Research  Simulator  (ERS)  as  a  major  tool  in  these  studies.  Some  examples 
of  past  efforts  include  Defensive  System  Upgrade  Program  (DSUP)  studies, 
Conventional  Mission  Upgrade  Program  (CMUP)  studies,  Block  Upgrade  studies,  Crew 
Station  Working  Group  (CSWG)  support,  a  Link  16  Study,  and  a  Baseline  Study 
(Kalman,  Kline,  Provost,  Gable,  &  Taylor,  1999).  Government  engineers  have  worked 
closely  with  the  contractors,  Boeing  (B-1B)  and  L3  Communications  (formerly  Raytheon 
Training  Systems)  (CSEF),  to  support  these  applied  human  factors  engineering  efforts 
and  to  help  fold  the  results  back  into  the  B-1B  system  to  make  it  a  safer  and  more 
effective  weapon  system. 

New  smart  weapons  and  advanced  avionics  systems  have  spurred  the  need  for  B-1B 
upgrades.  The  current  Block  D  upgrade  provides  the  B-1B  the  capability  to  carry  24 
Joint  Direct  Attack  Munitions  (JDAMs).  The  JDAM  tail  kit  turns  a  2000-pound  dumb 
bomb  (a.k.a.  MK-84)  into  a  global  positioning  system  (GPS)  aided  inertial  navigation 
system  (INS)  guided  near  precision  weapon.  This  increase  in  weapon  capabilities  also 
increases  mission  complexity.  Before  Block  D,  the  B-1B  would  drop  multiple  bombs  on 
a  single  target  area.  Now  the  B-1B  has  the  capability  to  direct  individual  weapons  to 
individual  targets.  Since  a  JDAM  is  a  guided  weapon,  the  weapon  launch  point  has 
expanded  into  a  region.  The  launch  acceptability  region  (LAR)  is  the  area  from  which  the 
JDAM  must  be  launched  in  order  to  achieve  the  desired  mean  point  of  impact  (DMPI). 
The  mission  complexity  arises  when  multiple  DMPIs  are  planned  resulting  in  multiple 
LARs.  The  proposed  Block  E  upgrade  incorporates  advanced  weapons  delivery 
computers,  terrain  following  computers,  guidance/navigation  computers,  and 
controls/displays  computers.  These  upgrades  will  be  critical  for  crewmember  situational 
awareness  in  the  JDAM  environment.  The  ALT  LAR  effort  was  based  on  the 
recommendations  made  in  the  B-1B  Human  Factors  Baseline  Study  Report  (Kalman,  et 
ah,  1999).  This  study  indicated  the  need  for  improvements  to  the  Block  E  LAR  displays. 
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3.0  STUDY  OBJECTIVE 


The  objective  of  this  study  was  to  perform  a  human  factors  evaluation  of  B- 
Alternative  Launch  Acceptability  Region  Displays  and  make  recommendations  to  B- 
System  Program  Office  Engineering  (SPO)  (ASC/YDE). 
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4.0  METHOD 


4.1  Subjects 

a.  Five  B-1B  Crews  -  Pilot,  Copilot,  Offensive  System  Operator  (OSO)  and 
Defensive  System  Operator  (DSO)  (i.e.,  20  total  subjects)  were  scheduled.  Due  to 
conflicts,  two  of  the  five  crews  were  only  able  to  supply  one  pilot  for  the  study. 
As  a  result,  a  total  of  ten  Weapon  System  Operators  (WSO)  and  eight  pilots 
participated  in  the  study. 

b.  ASC/YDE  was  responsible  for  supplying  the  subjects. 

c.  The  crews  were  on  station  at  the  CSEF  for  approximately  1  day  to  participate 
in  the  ALT  LAR  study.  Pilots  and  copilots  stayed  an  additional  day  to  participate 
in  the  Radar  Warning  Receiver  (RWR)  Display  study. 

4.2  B-1BERS 

The  study  was  performed  at  Wright-Patterson  AFB,  Ohio,  Area  B,  Building  145, 
Simulation  Bay  2  and  used  the  B-1B  Engineering  Research  Simulator  (ERS).  No 
hardware  changes  to  the  B-1B  ERS  were  necessary  for  this  study. 

4.3  Subject  Training 

Crews  were  given  an  introduction  briefing  covering  the  study’s  purpose, 
procedures,  and  schedule.  Each  crew  was  given  B-1B  ERS  familiarization  time  to 
fly  practice  missions  and/or  a  series  of  mission  segments  before  data  collection 
began.  Data  collection  missions  were  briefed  prior  to  flight. 

4.4  Missions 

There  was  one  type  of  Data  Collection  mission  (see  figure  1).  The  Data 
Collection  mission  was  based  on  the  mission  from  the  B-1B  Human  Factors 
Baseline  Study  (Kalman,  et  ah,  1999).  The  mission  was  flown  twice  with  the 
ALT  LAR  displays,  with  crewmembers  changing  seats  between  missions  (e.g., 
DSO  to  OSO’s  seat).  The  mission  contained  threats,  hung  stores,  and  forced 
deviations  from  the  “black  line”  or  planned  route.  The  mission  objective  was  to 
navigate  through  3  planned  target  areas  that  included  22  JDAM  targets.  The 
crews  were  instructed  to  use  standard  operating  procedures  for  threat  avoidance. 

4.5  ALT  LAR  Displays 

The  primary  displays  for  determining  LAR  location  are  found  on  the  OSO’s 
Multi-Function  Display  (MFD).  The  Target  Summary  display  (E  page)  is  the  top- 
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level  target  format.  The  target  summary  display  is  made  up  of  eight  data  fields. 
Fields  one  through  seven  are  repeated  for  each  target  summary  displayed.  Field 
eight  is  repeated  for  each  set  of  bay  doors.  Each  field  is  defined  as  follows: 


1.  Event  5. 

2.  Target  Type  6. 

3.  Target  Bearing  7. 

4.  Time-To-Go  8. 


Achievable  Zone/Range 
Range/Zone  Status 
Weapons  Available/Required 
Door/Position 


The  LAR  Status  Display  (EB  page)  provides  the  OSO  with  a  detailed  status  of 
each  near-mode  LAR  target.  The  data  for  up  to  eight  targets  can  be  presented  on 
one  page.  The  shaded  bar  indicates  the  location  and  width  of  each  target’s  in-zone 
LAR,  relative  to  current  aircraft  parameters  (see  figure  2).  The  top  and  bottom 
lines  represent  the  location  and  width  for  in-range  LARs.  An  arrow  on  the  left  or 
right  side  indicates  additional  target  data  extends  beyond  the  display  area. 


The  ALT  LAR  displays  contained  the  following  features  (see  figures  2  and  3): 


a.  There  was  an  improved  EB  page,  which  included: 

1 .  Angular  representation  of  LAR  bar  in  degrees 

2.  Current  Magnetic  Heading 

3.  Release  Time-To-Go  (TTG) 

i.  When  not  heading  towards  LAR,  times  are  estimated 

ii.  When  in  LAR,  TTG  reads  “LAR” 

4.  Available  Time  in  LAR  (seconds) 

i.  When  in  LAR,  Time  in  LAR  starts  to  count  down 

(99  seconds  is  default  maximum  value  for  Time  in  LAR  ) 

5.  OSO  controlled  mag-heading  bug 

i.  Can  be  connected  to  autopilot 

ii.  Actual  number  drives  bug,  line  is  just  a  visual  reference 

6.  LARs  show  a  “fly-into”  orientation,  the  bottom  LAR  bar  is  the  active 
weapon  and  successive  weapons  stack  upward.  (Current  Block  E  has  a 
top-down  order  of  release.) 

7.  Heading  to  DMPI 

b.  Estimated  TTG  was  provided  on  E  Page 

c.  OSO  Track  Handle  control  of  LAR  Bar  Page  functions  (see  figure  4) 

1.  Declutter/Scale  Field  of  View 

2.  Heading  Slew 

3.  Page  Scroll 

4.  Heading  Mode  Enable/Disable 

d.  Map  Enable  function  was  incorporated  (see  figure  5) 

1.  Located  in  OSOs  upper  right  MFD 

2.  Data  included: 

i.  Course  line 

ii.  Navigation  points 

iii.  DMPI  location 

iv.  LAR  location 
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Offensive  System  Operator  (OSO)  controlled  LAR  Steering  cue  or  bug  (e.g., 
“chicken’s  foot”)  on  pilot  and  copilot’s  Vertical  Situation  Display  (VSD) 


Figure  1.  ALT  LAR  Mission 
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Angular  representation  of 
LAR  Bar  in  degrees. 


Current  Mag  Heading. 


Time-To-Go/Time  in  LAR  (sec) 
(Note:  when  not  heading  towards 
LAR,  times  are  estimated).  Default 
max  value  for  time  in  LAR  is  99 
seconds. 


OSO  controlled  mag-heading  bug,  „ 
can  be  coupled  to  autopilot. 

Actual  number  drives  bug,  line 
is  just  a  visual  reference. 

LARs  show  a  “fly-into”  orientation, 
the  bottom  LAR  is  the  first  to  drop. 


B/  i  37L 

n  j  j  t 

Ut  ]?ni0  .5 


EVEN  f / 
L1BH— H— UH 
01=17/75 
uau-u-u,1 


Figure  2.  ALT  LAR  EB  Page 


D3  Mods 

Replace  Track  (TK)  with  > 
Magnetic  Course  (MC). 
Replace  True  Heading  (TH) 
with  the  Digital  Bullseye. 


When  in  LAR  TTG  _ 
reads  “LAR,”  and  time 
in  LAR  starts  to  count 
down. 


Hr' '/+! 

OB0-9  L'l  11 

1  NL, 

BIST 

HE  311,5-  5‘33 

1  1  D  1  -■  ■  ' 

|.|,  11'  iv  i  I  I 

urn  i  ri11  ms,  5 13 

ETfl 

bawzh  it/0 

■r  ■  ■  i 

ra  i  o*1  i 

nr  :  :: 

r  E  in  30  Li  Pi  11.  H 

-3‘j  -10  'j  795  0  to  t'j 

1  1  1 

WtMf-B-Ml 

LflR 


Lfl  ft 

/EE 

HQ- 

B-0, 

LflR 

/Si 

»l  rl  N 

11 -fll 

Lfl  R 

/El 

293 . 3 


Figure  3.  Decluttered  ALT  LAR  EB  Page 


OSO  Track  Handle 


Track  Handle  Functions 

1:  Declutter/Scale  Field  of  View 
2:  Fleading  Slew 
3:  Page  Scroll 

4:  Fleading  Mode  Enable/Disable 


Figure  4.  OSO  Track  Handle 


Figure  5.  Map  Enable  Page 


9 


5.0  DATA  COLLECTION 


5.1  The  following  data  were  collected: 

a.  Workload  -  Subjective  Workload  Assessment  Technique  (SWAT) 

b.  Questionnaire  -  A  questionnaire  was  given  to  obtain  the  WSO’s 
subjective  ratings  and  comments  (See  Appendix  1). 

c.  Video  Tape  -  The  OSO  and  Pilot  were  video  taped  during  Data  Collection 
missions 

d.  Number  of  bombs  dropped  within  LAR 

e.  Duration  and  frequency  of  OSO  utilization  of  the  EB  page  and  Map  Enable  on 
MFDs 

5.2  SWAT  data  was  collected  during  three  pre-planned  mission  freeze  points.  The  sole 
purpose  for  the  freeze  points  was  to  collect  SWAT  data. 
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6.0  DATA  ANALYSIS 


6.1  Measures  of  Workload. 

SWAT  is  used  to  assess  mental  workload  by  making  relative  comparisons  of  task 
conditions.  SWAT  is  an  accepted  technique  for  determining  if  a  task  under  one 
condition  requires  a  greater  mental  workload  than  under  another  condition.  SWAT 
was  handled  IAW  "SUBJECTIVE  WORKLOAD  ASSESSMENT  TECHNIQUE 
(SWAT):  A  USERS  GUIDE  (U)"  AAMRL-TR-89-023,  (Reid,  Potter,  &  Bressler, 
July  1989).  SWAT  Version  3.1  was  used  to  aid  in  the  computer  processing  of  the 
SWAT  data.  SWAT  ratings  are  broken  down  into  three  parts:  Time  Load,  Mental 
Effort  Load,  and  Psychological  Stress  Load.  During  Training,  the  Crew 
accomplished  the  SWAT  Card  Sort.  Only  SWAT  data  was  analyzed  statistically  by 
SPSS. 

6.2  Questionnaires. 

Questionnaire  data  is  summarized  by  using  average  ratings  and  response  frequency 
distributions  (if  appropriate)  (See  Appendix  1).  Subject's  comments  are  summarized. 

6.3  There  was  no  statistical  comparison  between  B-1B  Block  E  LAR  and  the  ALT  LAR 
SWAT  scores. 

The  ALT  LAR  SWAT  Scores  were  compared  to  the  SWAT  score  of  40  which  is 
regarded  as  the  borderline  score  between  acceptable  workload  (e.g.,  a  score  of  20) 
and  unacceptable  workload  (e.g.,  a  SWAT  score  of  50)  (see  figure  6). 

6.4  All  missions  were  video  taped. 

6.5  The  number  of  bombs  dropped  within  LAR  was  scored  and  analyzed  (see  table  1). 

6.6  SWAT  data  analysis. 

Figure  6  displays  the  mean  SWAT  values  at  each  freeze  point.  At  all  freeze  points 
SWAT  exceeded  the  redline  value  of  40.  As  expected  the  highest  workload  was 
associated  with  freeze  point  2  (67.53)  and  the  least  with  freeze  point  1  (52.01). 
However,  there  was  no  significant  (p>.05)  relationship  between  SWAT  and  freeze 
points.  Likewise,  adding  OSO  experience  to  the  model  again  showed  no  significant 
differences  (see  Appendix  2).  Given  no  significant  main  effects  or  interactions, 
further  analysis  was  not  warranted  for  SWAT. 
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Figure  6.  SWAT  Ratings  Compared  to  40 


6.7  Bombs  on  target. 

The  ALT  LAR  mission  was  pre-planned  with  24  JDAM  available.  Of  these,  a  total  of 
22  DMPI  locations  were  assigned  leaving  2  JDAM  unassigned.  During  the  first 
bomb  run,  a  planned  hung  store  occurred  on  an  outer  launcher  location  blocking  two 
inner  JDAM.  As  a  result,  a  total  of  three  JDAM  were  unusable  leaving  a  total  of  2 1 
available  JDAM.  Due  to  JDAM  variances  in  the  bays,  the  automatic  reallocation 
function  was  limited  forcing  the  OSO  to  manually  reallocate  in  order  to  achieve 
100%  mission  success.  The  mission  included  a  forced  deviation  from  the  planned 
route  or  “black  line”  due  to  threat  avoidance.  Table  1  depicts  the  bomb  success  rate 
per  subject. 


Table  1.  Number  of  Bombs  in  LAR 


CREW 

JDAMS 

DROPPED 

ON 

BLACK 

LINE 

OFF 

BLACK 

LINE 

MSN  SUCCESS 
RATE  IN  % 

NOTES 

1 

20 

14  OF  15 

6  OF  6 

95.2 

OSO  elected  not  to  reallocate  hung 
weapons 

2 

19 

14  OF  14 

5  OF  7 

90.5 

Deviation  from  black  line  caused  short 
time  in  LAR,  OSO  had  good  SA 

3 

18 

12  OF  14 

6  OF  7 

85.7 

Hung  store  task  saturation,  OSO  slow 
to  use  available  resources  (i.e.  EB 
page,  Map  Enable) 

4 

20 

14  OF  14 

6  OF  7 

95.2 

Slow  to  reallocate  caused  fly  through 
LAR  before  all  weapons  released 

5 

21 

14  OF  14 

7  OF  7 

100 

6 

20 

14  OF  15 

6  OF  6 

95.2 

OSO  elected  not  to  reallocate  hung 
weapons 

7 

21 

14  OF  14 

7  OF  7 

100 

8 

13 

12  OF  14 

1  OF  7 

61.9 

OSO  lost  SA  during  hung  store 
reallocation  and  flew  through  LARs, 

9 

21 

14  OF  14 

7  OF  7 

100 

10 

21 

14  OF  14 

7  OF  7 

100 

MEAN 

SUCCESS 

RATE 

95.77 

85.29 

92.4% 

6.8  OSO  utilization  of  EB  page  and  Map  Enable  function  (see  table  3). 

There  was  no  correlation  between  SWAT  and  the  two  dependent  measures  of  percent 
of  time  on  the  EB  page  and  the  percent  of  time  on  Map  Enable  (see  Appendix  2). 
The  two  dependent  measures,  EB  percent  and  Map  Enable  percent,  were  not 
significantly  correlated  (p>.05)  with  each  other,  requiring  that  they  be  analyzed  in 
separate  designs.  As  shown  in  figure  7,  the  time  utilization  percentages  are  fairly 
uniform  for  EB  page.  However,  for  Map  Enable  there  is  great  disparity  in  percentage 
between  freeze  point  one  and  the  other  freeze  points.  This  is  reflected  in  the  separate 
repeated  measures  analysis.  For  both  analyses,  experience  was  kept  in  the  model. 
Although  the  interaction  of  freeze  point  and  experience  was  not  significant  (p>.05), 
keeping  experience  in  the  models  raised  the  predictive  value  of  the  freeze  point  main 
effect  (eta-square)  from  .237  to  .411  for  EB  page  and  .412  to  .750  for  map  enable. 
Even  with  experience  in  the  model,  there  is  no  significance  (p>.05)  for  the  main 
effect  of  freeze  point  with  EB  page  percentage  as  the  dependent  variable.  As  in  the 
case  of  SWAT,  further  analysis  was  not  warranted.  On  the  other  hand,  significance 
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was  found  (p<.05)  when  the  dependent  variable  was  map  enable  percentage  as 
displayed  in  table  2. 


50 
45 
40 
35 

AVERAGE 
PERCENTS  20 

15 
10 
5 
0 

1  2  3  TOTAL 

FREEZE  POINT 


□  EB  Page 
BMap  Enable 


Figure  7.  Average  Utilization  of  EB  Page  and  Map  Enable 


Table  2.  Multivariate  Effect  of  Freeze  Point  with  Experience  on  Map  Enable  Percent  of 

Time 


Effect 

Value 

F 

Hypothesis 

df 

Error  df 

Sig. 

Eta 

Squared 

FRZPT 

Pillai's  Trace 

.750 

7.505(a) 

2.000 

5.000 

CO 

O 

* 

.750 

Wilks'  Lambda 

.250 

7.505(a) 

2.000 

5.000 

.031 

.750 

Hotelling's 

Trace 

3.002 

7.505(a) 

2.000 

5.000 

.031 

.750 

Roy's  Largest 
Root 

3.002 

7.505(a) 

2.000 

5.000 

.031 

.750 

FRZ  PT* 
EXPERBIN 

Pillai's  Trace 

.748 

1.194 

6.000 

12.000 

.372 

.374 

Wilks'  Lambda 

.256 

1.628(a) 

6.000 

10.000 

.236 

.494 

Hotelling's 

Trace 

2.893 

1.929 

6.000 

8.000 

.191 

.591 

Roy's  Largest 
Root 

2.889 

5.777(b) 

3.000 

6.000 

.033 

.743 

a  Exact  statistic 

b  The  statistic  is  an  upper  bound  on  F  that  yields  a  lower  bound  on  the  significance  level. 

c  Design:  Intercept+EXPERBIN  *p<.05 

Within  Subjects  Design:  FRZ PT 
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Table  3.  OSO  Utilization  of  “EB  Page”  and  Map  Enable 


CREW 

BOMB 

RUN 

TIME  ON 

EB  PAGE  (SEC) 

%  TIME  ON 

EB  PAGE 

TIME  ON  MAP 
ENABLE  (SEC) 

%  TIME  ON 
MAP  ENABLE 

1 

1 

155 

16.4 

217 

22.9 

2 

121 

21.5 

32 

5.7 

3 

103 

39.9 

3 

1.2 

2 

1 

242 

25.6 

61 

6.5 

2 

220 

39.0 

68 

12.1 

3 

136 

52.7 

32 

12.4 

3 

1 

392 

41.4 

141 

14.9 

2 

101 

17.9 

0 

0 

3 

98 

38.0 

2 

0.8 

4 

1 

304 

32.1 

770 

81.4 

2 

40 

7.1 

0 

0 

3 

30 

11.6 

0 

0 

5 

1 

428 

45.2 

826 

87.3 

2 

199 

35.3 

2 

0.4 

3 

119 

86.2 

0 

0 

6 

1 

600 

63.4 

370 

39.1 

2 

481 

85.3 

60 

10.6 

3 

145 

56.2 

123 

47.7 

7 

1 

561 

59.3 

156 

16.5 

2 

111 

19.7 

19 

3.4 

3 

49 

19.0 

10 

3.9 

8 

1 

295 

31.2 

39 

4.1 

2 

112 

19.9 

70 

12.4 

3 

258 

100.0 

109 

42.3 

9 

1 

787 

83.2 

475 

50.2 

2 

356 

63.1 

58 

10.3 

3 

103 

39.9 

44 

17.1 

10 

1 

556 

58.8 

9 

1.0 

2 

332 

58.9 

95 

16.8 

3 

87 

33.7 

0 

0 

AVERAGE 
SCORES 
PER  RUN 

1 

432  sec 

45.66% 

306.4  sec 

32.39% 

2 

207.3  sec 

36.77% 

40.4  sec 

7.17% 

3 

1 12.8  sec 

47.722% 

32.3  sec 

12.54% 

6.9  Questionnaire  data  analysis  (see  Appendix  1) 

All  study  participants  were  asked  to  fill  out  a  questionnaire  after  completing  the  study 
missions.  All  questionnaires  contained  demographic  information  followed  by  a  pilot 
only,  or  WSO  only,  section. 
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All  rating  scales  were: 

7  Totally  Acceptable 
6  Very  Acceptable 
5  Somewhat  Acceptable 
4  Borderline 

3  Somewhat  Unacceptable 
2  Very  Unacceptable 
1  Totally  Unacceptable 

All  question  ratings  were  averaged  across  crewmembers.  This  number  determined 
the  final  rating.  A  rating  frequency  histogram  is  in  Appendix  1  with  each  question. 

All  crew  comments  are  reported  in  Appendix  1  verbatim. 
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7.0  DISCUSSION 


7.1  SWAT  discussion. 

The  B-1B  ERS  is  a  real  time,  high  fidelity  system,  however,  realistic  combat 
mission  conditions  are  not  feasible  for  the  study.  For  example,  there  is  no  visual 
system,  which  greatly  reduces  pilot  procedures,  and  there  is  no  aircraft 
communication  with  external  agencies  (e.g.,  AW  ACS,  Command  Post).  In  order 
to  achieve  a  valid  experimental  design,  OSO  workload  was  artificially  raised 
through  the  use  of  a  pre-planned  hung  store  during  the  first  bomb  run  and  forced 
deviations  from  the  “black  line”  due  to  threat  avoidance.  The  SWAT  data  was 
collected  at  three  pre-determined  freeze  points  throughout  the  mission  in  order  to 
validate  the  OSO  workload  designed  into  the  study.  The  Air  Force  Research 
Faboratory  (the  SWAT  developer)  has  determined  that  a  group  mean  score  of  40 
represents  a  “red  line”  (i.e.,  it  is  an  indicator  that  performance  may  start  to 
degrade).  As  the  SWAT  data  shows,  the  OSO  workload  for  the  mission  was  over 
the  “red  line”  for  each  mission  segment  with  freeze  point  two  showing  the  highest 
average  score  of  67.63.  Since  the  manual  reallocation  of  weapons  occurred 
mostly  between  freeze  points  one  and  two,  the  highest  SWAT  scores  were 
expected  for  this  mission  segment. 

Further  analysis  shows  that  there  was  no  correlation  between  OSO  experience  and 
workload  levels.  Consequently,  workload  was  determined  to  be  high  for  the 
planned  mission  for  all  OSOs.  It  is  important  to  understand  that  high  workload  is 
an  intended  effect.  In  a  low  workload  environment,  we  would  expect  a  very  high 
mission  success  rate.  In  a  high  workload  environment  we  expect  OSO 
performance  to  deteriorate.  The  OSOs  were  placed  into  an  artificially  high 
workload  environment  in  order  evaluate  performance  under  these  simulated 
conditions. 

7.2  OSO  performance. 

In  a  high  workload  environment,  the  OSOs  were  able  to  average  a  bombs-on- 
target  rate  or  mission  success  rate  of  92.4%.  A  total  of  four  crews  achieved  a 
100%  success  rate  and  only  two  crews  scored  below  90%.  These  statistics 
provide  a  favorable  objective  measure  of  the  AFT  FAR  software.  When 
workload  is  increased,  the  OSO  only  experienced  a  7.6%  average  degradation  in 
performance.  Breaking  down  bombing  statistics  with  respect  to  deviations  from 
the  pre-planned  route  or  “black  line,”  the  results  show  95.77%  accuracy  when  on 
the  “black  line”  and  85.77%  when  off  the  “black  line.”  Experiencing  only  a  10% 
decrease  in  performance  when  major  deviations  occurred  from  planned  route  is 
another  favorable  objective  measure  supporting  the  AFT  FAR  displays.  Also,  a 
review  of  the  mission  tapes  shows  that  several  OSOs  maintained  situational 
awareness  and  realized  that  they  were  going  to  exit  FAR  before  all  weapons  were 
released  due  to  launcher  rotation  limitations. 

As  a  subjective  measure,  the  questionnaires  also  show  strong  support  for  the  AFT 
FAR  information.  The  AFT  FAR  changes  to  current  block  E  displays  were  rated 
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“very  acceptable”  scoring  6.2  on  a  scale  of  1  to  7.  Each  individual  modification 
was  also  rated  for  its  acceptability.  The  modifications  receiving  a  “very 
acceptable”  rating  (5.6  to  6.5)  include:  heading  mode  to  provide  steering 
commands,  LAR  bar  display,  implementation  of  the  track  handle,  declutter  mode, 
all  scale  modes  of  the  LAR  bar  display,  time-to-go/time-in-LAR  information  on 
the  EB  page,  angular  representation  of  LAR  bar  page,  and  magnetic  heading 
information  on  the  LAR  bar  page. 

The  OSOs  also  rated  the  operational  utility  of  the  ALT  LAR  displays.  Ability  to 
line  up  multiple  LARs  and  target  situational  awareness  (both  on  and  off  black 
line)  received  a  “very  acceptable”  rating.  Inter-crew  coordination  only  received  a 
somewhat  acceptable  rating. 

Analysis  of  the  OSO  utilization  of  the  EB  page  also  shows  favorable  support  for 
the  ALT  LAR  changes.  There  was  a  43.38%  average  utilization  rate  per  bomb 
run  of  the  EB  page.  However,  since  the  EB  page  is  primarily  designed  as  a  target 
guidance  system,  this  actually  constitutes  a  151%  of  the  pre-planned  IP-to-target 
mission  segment  times.  Observation  of  the  mission  tapes  shows  that  most  OSOs 
elected  an  early  sequence  to  the  target  waypoint  in  order  to  gain  full  functionality 
of  the  EB  page. 

The  track  handle  implementation  received  strong  support.  However,  as 
implemented  in  the  ERS,  there  was  some  concern  over  the  heading  mode 
functionality.  When  in  heading  mode,  there  was  no  indication  at  the  pilot’s 
station  that  heading  mode  was  selected.  Also,  a  conscious  effort  on  the  OSO’s 
part  was  required  to  deselect  heading  mode  when  no  longer  needed.  This  resulted 
in  deviations  from  planned  route  of  flight  on  several  occasions. 

7.3  Other  OSO  modifications. 

This  study  incorporated  two  modifications  separate  from  the  ALT  LAR  changes. 
The  first  modification  was  the  implementation  of  the  Map  Enable  function  (see 
figure  5).  This  function  allowed  the  OSO  a  “God’s  eye  view”  of  the  mission. 
Typical  OSO  comments  on  the  Map  Enable  function  were  very  positive  (see 
Appendix  2): 

-  (Map  Enable  mode)  is  a  huge  SA  builder 

-  Map  Enable  really  helps  with  deviation.  Without  it,  it  would  be  more 
difficult  to  keep  SA 

-  Map  is  great  with  LARs 

-  The  Map  does  wonders  after  getting  distracted. . . 

One  area  of  concern  is  derived  from  analysis  of  the  utilization  rates  of  the  Map 
Enable  function.  There  is  a  significant  correlation  between  high  workload  and 
low  utilization  of  the  Map  Enable  feature  when  analyzed  across  experience  levels. 
This  finding  suggests  that  placing  the  function  on  one  of  the  two  MFDs  may  not 
be  an  optimum  solution.  The  OSOs  use  the  MFDs  for  navigation,  target 
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information,  and  weapon  monitoring  during  the  bombing  ran.  Although  the  Map 
Enable  feature  positively  influences  situational  awareness,  it  restricts  use  of  as 
much  as  50%  of  available  MFD  information. 

The  second  modification  incorporated  changes  to  the  Nav  Prime  displays  on  the 
top  of  the  MFDs  (see  figure  3).  The  change  replaced  True  Track  and  True 
Heading  information  with  Magnetic  Course  and  Digital  Bullseye  information. 
The  questionnaire  data  rated  these  changes  as  “Totally  Acceptable”  (see 
Appendix  1). 

7.4  Pilot  questionnaires. 

The  only  modification  to  the  pilots  station  for  this  study  was  the  implementation 
of  the  heading  cursor  or  “chicken  foot.”  This  modification  allowed  the  OSO  to 
give  steering  commands  through  his  MFD  to  the  pilot.  According  to 
questionnaires,  the  pilots  rated  targeting  information  as  “Somewhat 
Unacceptable”  (See  Appendix  1).  The  two  other  factors  analyzed,  target 
situational  awareness  (both  on  and  off  “black  line”)  and  inter-crew  coordination, 
were  also  rated  as  “Somewhat  Unacceptable.”  Most  comments  indicated  the  need 
for  more  information: 

-  After  a  threat  reaction,  dismal  SA  becomes  abysmal 

-  Pilots  have  almost  no  SA  after  deviations 

-  Situational  awareness  inputs  are  less  than  optimum  because  they  rely  on  verbal 
inputs  from  the  OSO  rather  than  a  visual  presentation 
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8.0  RECOMMENDATIONS 


8. 1  ALT  LAR  recommendations. 

Control  and  display  improvements  to  the  current  Block  E  LAR  displays  are 
recommended  to  increase  the  OSO’s  situational  awareness  and  improve 
performance. 

a.  Time-to-go  and  time-in-LAR  information  should  be  incorporated  into  the 
EB  page.  This  change  will  greatly  reduce  cycling  between  MFD  pages. 

b.  A  scalable  angular  representation  of  the  LAR  bars  with  a  fly-through 
orientation  (bottom-to-top)  should  be  incorporated.  This  will  improve 
OSO  situational  awareness  by  showing  relative  distance  to  the  LARs. 

c.  Estimated  time-to-go  information  needs  to  be  included  when  in  near-mode 
ranging  and  not  heading  towards  LAR.  Technical  Manual  information 
should  note  that  this  information  may  be  inaccurate  but  will  update  real 
time  as  the  aircraft  heading  approaches  LAR  intercept  heading. 

d.  Current  aircraft  magnetic  heading  should  be  included  on  EB  page. 

e.  Track  Handle  control  of  EB  page  functions  would  improve  OSO 
performance.  Further  analysis  of  the  heading  mode  function  and  “chicken 
foot’ ’  control  is  warranted. 


8.2  Other  OSO  modifications. 

a.  The  Map  Enable  function  or  similar  format  should  be  incorporated  into  the 
OSO  station.  This  improvement  will  greatly  increase  OSO  situational 
awareness.  Serious  consideration  should  be  given  to  finding  an  alternate 
location  than  on  one  of  the  two  MFDs. 

b.  Changes  to  the  Nav  Prime  displays  should  be  implemented.  This 
modification  had  the  strongest  support  from  the  OSOs  in  the  study. 


8.3  Pilot  station  recommendations. 

ilot  situational  awareness  in  terms  of  LAR  is  unacceptable.  Current  displays 
should  be  improved  to  include  LAR  information  and  current  track  line. 
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APPENDIX  1.  QUESTIONNAIRES 


B-1B 

Alternative  Launch  Acceptability  Region  (ALT  LAR)  Study 

Questionnaire 

1.  A.  Pilot/Copilot  Experience 

Current  Qualified  Aircraft: _  Number  of  Flying  Hrs _ 

B-  IB  Flight  Hours:  Average:  1340  Min:  260  Max:  2300 

Other  Aircraft  Flown: _  Number  of  Flying  Hrs _ 

Other  Aircraft  Flown: _  Number  of  Flying  Hrs _ 

Other  Aircraft  Flown: _  Number  of  Flying  Hrs _ 

Other  Aircraft  Flown: _  Number  of  Flying  Hrs _ 

Total  Flying  Hrs: _ 

1.  Total  Flight  Hours:  Average:  2086  Min:  470  Max:  3600 

2.  Have  you  had  previous  B- IB  Block  D?  Yes_7_  No  1 

If  yes,  what  aircraft/simulator(s)?  Sim-1.  A/C-6  Number  of  Hrs  Ave:  79 

Min:  10  Max:  200 


3.  Have  you  had  any  training  on  JD AM?  Yes  7  No  1_ 

4.  Are  you  familiar  with  Launch  Acceptability  Regions  (LARs)?  Yes  7  No  1 

5.  Have  you  ever  participated  in  any  B-1B  studies  or  flight  tests?  Yes_5_  No  3_ 

If  yes,  describe:  ALR-56M  Repeater,  ERS  Baseline,  Block  D  IOT&E,  Block  D 
FDE,  PT  EPW,  OT&E,  TD&E,  FME,  ILS  display,  VSD  upgrade. 
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Pilot  Section: 

1.  Rate  the  acceptability  of  the  LAR  heading  display  (chicken  foot)  and  targeting 
information.  Use  the  following  scale: 


Ratings: 

7  Totally  Acceptable  0 

6  Very  Acceptable  0 

5  Somewhat  Acceptable  0 

4  Borderline  3 

3  Somewhat  Unacceptable  2 

2  Very  Unacceptable  1 

1  Totally  Unacceptable  2 

RATING: _  Average:  3.38 


Comments: 

•  I  need  to  have  LAR  displays  up-front. 

•  Pilots  need  display  of  LAR  -  moving  map  with  threat  and  LAR  display  would  be 
great! ! 

•  Need  message  on  VSD  indicating  OSO  is  steering  the  LAR,  i.e.,  "LAR  STRG." 

•  Need  some  way  to  let  pilots  know  which  steering  mode  the  OSO  is  in.  Recommend 
CDI  stay  connected  to  Nav  point  and  just  heading  bug  move. 

•  Steering  commands  are  fine,  however  the  LAR  heading  display  alone  does  not 
provide  sufficient  situational  awareness  to  allow  the  pilot  to  place  the  aircraft  in  the 
LAR. 

•  Targeting  information  is  only  available  at  pilot  station  if  the  pilots  program  each 
DMPI  into  the  CDU,  but  this  is  only  info  to  go  DMPI  direct.  The  info  from  the  CDU 
does  not  indicate  heading  to  LAR  entry  center  lead  point. 

•  The  chickenfoot  was  fine,  however,  targeting  information  was  also  adequate. _ 
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2.  Were  you  able  to  use  the  LAR  Heading  Display  to  steer  to  the  upcoming  weapons 
I.AR(s)? 

Yes  7  No  1  Comments: 


•  Need  some  indicator/reverse  video  up-front  for  when  OSO  goes  into  LAR  steering 

•  Chickenfoot  and  voice  directions  are  insufficient  in  a  task  saturation  and  comm. 
Intensive  environment. 

•  There  is  no  LAR  steering  in  the  front  station 

•  Pilots  need  some  type  of  "God's  Eye"  view  with  LAR  superimposed. 

•  The  pilot  can  manually  turn  beyond  30°  bank  to  reach  the  LAR  with  positive  SA 
only  when  prompted  by  the  OSO  after  the  OSO  selected  heading  mode. 

•  Easily _ 
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3.  Rate  the  acceptability  of  the  overall  Target  Situational  Awareness.  Use  the  following 


scale: 

Ratings: 

7  Totally  Acceptable  0 

6  Very  Acceptable  1 

5  Somewhat  Acceptable  2 

4  Borderline  2 

3  Somewhat  Unacceptable  3 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  3.50 


Comments: 


•  I  need  to  know  where  the  LARs  are  at  in  relation  to  me  and  the  target!  I  need  to  be 
able  to  back  up  WSOs  without  having  to  garbage  up  the  intercom. 

•  Pilots  need  better  early-late  indication;  perhaps  a  digital  readout  of  exact  timing  like 
WSO  stations. 

•  Have  no  target  situational  awareness. 

•  Pilots  need  some  type  of  "God's  Eye"  view  with  LAR  superimposed. 

•  Situational  awareness  inputs  are  less  than  optimum  because  they  rely  on  verbal  inputs 
from  the  OSO  rather  than  a  visual  presentation.  The  best  target  situational  awareness 
would  include  a  dynamic  LAR  display  with  LAR  entry/exit  countdown  displays. 

•  The  OSO  has  a  LAR  display  with  current  trackline,  so  he  must  vocalize  time-to-LAR 
and  turn  direction.  In  a  communications/threat  intensive  environment,  this  will  be 
difficult  without  a  repeater  scope  at  the  pilot  station  displaying  both  time-to-LAR  and 
LAR  position/direction.  The  pilot  station  needs  a  visual  indicator  to  build  SA  to  LAR. 

•  I  thought  target  SA  would  be  good. _ 
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4.  Rate  the  acceptability  of  the  Target  Situational  Awareness  with  respect  to  deviations 
from  the  planned  route.  Use  the  following  scale: 


Ratings: 

7  Totally  Acceptable  0 

6  Very  Acceptable  1 

5  Somewhat  Acceptable  2 

4  Borderline  0 

3  Somewhat  Unacceptable  1 

2  Very  Unacceptable  2 

1  Totally  Unacceptable  2 

RATING: _  Average:  3.13 


Comments: 


•  After  a  threat  reaction,  dismal  SA  becomes  abysmal.  I  need  to  know  where  the  LARs 
are  at  in  relation  to  me  and  the  target!  I  need  to  be  able  to  back  up  WSOs  without 
having  to  garbage  up  the  intercom. 

•  Pilots  have  almost  no  SA  after  deviations.  We  need  to  independently  steer  to  a 
displayed  LAR. 

•  No  target  displays. 

•  Pilots  need  some  type  of  "God's  Eye"  view  with  LAR  superimposed. 

•  Situational  awareness  inputs  are  less  than  optimum  because  they  rely  on  verbal 
inputs  from  the  OSO  rather  than  a  visual  presentation.  The  best  target  situational 
awareness  would  include  a  dynamic  LAR  display  with  LAR  entry/exit  countdown 
displays.  This  setup  requires  verbal  description  from  the  OSO.  Visual  presentation 
would  be  far  more  efficient 

•  Targeting  information  is  only  available  at  pilot  station  if  the  pilots  program  each 
DMPI  into  the  CDU,  but  this  is  only  info  to  go  DMPI  direct.  The  info  from  the  CDU 
does  not  indicate  heading  to  LAR  entry  center  lead  point. 

•  The  OSO  has  a  LAR  display  with  current  trackline,  so  he  must  vocalize  time-to-LAR 
and  turn  direction.  In  a  comm/threat  intensive  environment,  this  will  be  difficult 
without  a  repeater  scope  at  the  pilot  station  displaying  both  time-to-LAR  and  LAR 
position/direction.  The  pilot  station  needs  a  visual  indicator  to  build  SA  to  LAR. 

•  If  the  IP  is  near/after  the  LAR,  and  IP  D#  is  flown  to,  the  pilot  has  absolutely  no  SA 
that  he  has  passed  the  LAR  while  thinking  he  is  flying  IP  to  target  D#. 

•  Still  very  good. _ 
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5.  Rate  the  acceptability  of  the  Inter-Crew  Coordination  with  respect  to  LAR 
navigation.  Use  the  following  scale: 


Ratings: 

7  Totally  Acceptable  0 

6  Very  Acceptable  0 

5  Somewhat  Acceptable  3 

4  Borderline  2 

3  Somewhat  Unacceptable  1 

2  Very  Unacceptable  1 

1  Totally  Unacceptable  1 

RATING: _  Average:  3.36 


Comments: 


•  Doing  a  bomb  run  must  not  be  this  comm  intensive.  We  need  to  give  everyone  in  the 
jet  as  much  SA  as  possible  instead  of  designing  a  "Fve  got  a  secret"  cockpit  setup. 

•  Comm  breaks  down  in  a  comm  intensive  or  high  task  environment.  Need  a  birdseye 
view  of  LAR  at  front  station. 

•  Pilots  need  to  have  an  indication  of  when  the  OSO  has  taken  control  of  heading 
marker/chickenfoot  (reverse  video) 

•  Need  message  on  VSD  indicating  OSO  is  steering  the  LAR,  i.e.  "LAR  STRG" 

•  Pilots  must  know  the  type  of  steering  the  WSO  has. 

•  Situational  awareness  inputs  are  less  than  optimum  because  they  rely  on  verbal  inputs 
from  the  OSO  rather  than  a  visual  presentation.  The  best  target  situational  awareness 
would  include  a  dynamic  LAR  display  with  LAR  entry/exit  countdown  displays. 

This  setup  requires  verbal  description  from  the  OSO.  Visual  presentation  would  be 
far  more  efficient. 

•  Targeting  information  is  only  available  at  pilot  station  if  the  pilots  program  each 
DMPI  into  the  CDU,  but  this  is  only  info  to  go  DMPI  direct.  The  info  from  the  CDU 
does  not  indicate  heading  to  LAR  entry  center  lead  point. 

•  The  OSO  has  a  LAR  display  with  current  trackline,  so  he  must  vocalize  time-to-LAR 
and  turn  direction.  In  a  comm/threat  intensive  environment,  this  will  be  difficult 
without  a  repeater  scope  at  the  pilot  station  displaying  both  time-to-LAR  and  LAR 
position/direction.  The  pilot  station  needs  a  visual  indicator  to  build  SA  to  LAR. 

•  In  an  comm/threat  intensive  environment  the  OSO  must  verbalize  the  SA  off  of  his 
scope,  while  the  integration  of  CDU  DMPI  direct  information  does  not  allow  for 
maximum  threat  avoidance,  while  still  obtaining  LAR,  due  to  having  to  point  the  nose 
directly  at  the  DMPI  which  might  mean  entering  a  threat  WEZ. 

•  Very  good. _ 
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General  Comments: _ 

•  Dynamic  DOB  -  even  if  a  threat  quits  emitting,  need  to  have  some  sort  of  "save" 
function  so  we  continue  to  have  SA  on  a  threat  that  did  emit,  but  no  longer  is. 

•  Bullseye  readout  of  whatever  the  OSO  locks  up  on  radar  -  like  a  tanker. 

•  DSO  compass  rose  need  the  30°  increments  on  it  -  N,  030,  060,  E,  120  etc. 

•  Investigation  of  the  bank  release  angle  limitation  while  releasing  JDAMs,  if  we  can 
safely  release  in  10,  20,  or  30°  bank,  why  not  pull  restriction  from  T.O.  IF  you  must 
be  in  0°  bank,  limit  LAR  Heading  Display  "chicken-foot"  updates  to  outside  5  sec. 
from  LAR  like  is  done  during  GMTI  bombing. 

•  ILST  -  need  repeater  at  pilot-station  -  need  bullseye  readout  of  radar  contacts. 

•  Compatible  NVG  lighting. 

•  Moving  map  with  course,  LAR,  threat  (at  least  DOB)  overlay. _ 
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OSO  Section 


B.  WSO  Experience 


Current  Qualified  Aircraft: _ Number  of  Flying  Hrs _ 

B-1B  DSO  Hrs:  Ave:  604.  Min:  100.  Max:  1200  A/C  Block:  A-D 
B-1B  OSO  Hrs  Ave:  714.  Min:  100.  Max:  1600  A/C  Block:  A-D 


Other  Aircraft  Flown: 
Other  Aircraft  Flown: 
Other  Aircraft  Flown: 
Other  Aircraft  Flown: 


Number  of  Flying  Hrs 
Number  of  Flying  Hrs_ 
Number  of  Flying  Hrs_ 
Number  of  Flying  Hrs_ 


Total  Flying  Hrs:  Ave:  1992,  Min:  350,  Max:  3600 


1. 

2. 

3. 

4. 


Have  you  had  previous  B- IB  Block  D?  Yes_6_  No_4 

If  yes,  what  aircraft/simulator(s)?  A/C:  5,  Number  of  Hrs  Ave:  150 

Min:  20  Max:  300 


Have  you  had  any  training  on  JD AM?  Yes_9_  No  1_ 

Are  you  familiar  with  Launch  Acceptability  Regions  (LARs)?  Yes  10  No  0_ 

Have  you  ever  participated  in  any  B-1B  studies  or  flight  tests?  Yes  3  No_7 

If  yes,  describe:  Human  Factors  studies,  Block  D  flight  test,  Baseline,  Block  E/F 
and  D 
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1.  Rate  the  acceptability  of  the  LAR  displays  and  targeting  information.  Use  the 


following  scale: 

Ratings: 

7  Totally  Acceptable  1 

6  Very  Acceptable  7 

5  Somewhat  Acceptable  2 

4  Borderline  0 

3  Somewhat  Unacceptable  0 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  5.90 


Comments: 

•  During  bomb  runs  I  had  to  cycle  through  E,  EB  and  D  pages.  Switching  pages 
lowered  my  SA. 

•  No  Block  D  experience.  LAR  bars  with  steering  from  OSO  are  somewhat  useful,  but 
pilots  don't  know  what  is  driving  "chicken  foot"  -  steering  command  from  OSO  or 
DAS  &  steer  #. 

•  The  [EB]  LAR  bars  are  great  -  especially  when  WSO/OSO  has  familiarization  time 
with  formats. 

•  Very  good  compared  to  Block  D  -  much  more  situational  awareness  available. 

•  The  [EB]  page  is  very  helpful  and  SA  building.  In  general  data  on  the  MFD  displays 
go  from  top-to-bottom.  This  data  goes  from  bottom  to  top.  It  is  different,  but  I  believe 
the  WSO  can  adjust. 

•  The  pylon  function  is  a  huge  SA  builder. 

•  What  range  is  the  "God's  eye"  view?  Great  SA  however!  Far  [mode]  ranging 
information  is  good. 

•  I  would  suggest  taking  out  the  -8  (representing  the  target.)  It  is  unnecessary  because 

we  knew  it  is  a  target. _ 
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2.  Are  you  familiar  with  current  Block  E  LAR  displays? 

Yes  5_  No  5 

2  a.)  If  yes,  rate  the  operational  utility  of  the  ALT  LAR  changes  to  the  current  Block 
E  displays.  Use  the  following  scale. 


Ratings: 

7  Totally  Acceptable  2 

6  Very  Acceptable  2 

5  Somewhat  Acceptable  1 

4  Borderline  0 

3  Somewhat  Unacceptable  0 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  6.20 


Comments: 


•  Again,  become  more  familiar  with  new  EA  and  EB  and  DAC  formats  -  the  LAR 
display  is  superb. 

•  The  ALT  LAR  display  [EB]  should  go  from  top-to-bottom  v.  bottom-to-top.  All  data 
is  currently  read  top-to-bottom  on  all  our  MFB  pages. 

•  Need  to  put  ENBL/ALL  or  EA  page  to  greatly  minimize  hey  strokes  especially  in  a 
re-allocation  situation. 

•  Need  to  use  something  besides  "NONE"  for  a  non-allocated  weapon  use  "AVLB"  for 
available. 
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3.  Rate  the  acceptability  of  the  Heading  Mode  for  providing  steering  commands  for 
upcoming  weapons  LAR(s)  to  the  pilot.  Use  the  following  scale: 


Ratings: 

7  Totally  Acceptable  1 

6  Very  Acceptable  5 

5  Somewhat  Acceptable  3 

4  Borderline  1 

3  Somewhat  Unacceptable  0 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  5.60 


Comments: 


•  For  black-line  sorties  it  is  acceptable.  However,  during  threat  reaction,  it  is 
unacceptable.  The  E  page  provides  more  SA  since  it  provides  degrees  L  or  R. 

•  LAR  bars  with  steering  from  OSO  are  somewhat  useful,  but  pilots  don't  know  what 
is  driving  "chicken  foot"  -  steering  command  from  OSO  or  DAS  &  steer  #.  Give 
pilots  an  indication  on  VSD  as  to  which  mode  the  OSO  is  in. 

•  Good  option  from  at  least  4  minutes  out  from  LAR  release. 

•  WSO/OSO  need  to  remember  to  go  to  pilot  nav.  heading! 

•  It  is  good  except  there  isn’t  any  indication  to  the  crew  that  this  is  engaged.  Need 
some  kind  of  indication  to  the  pilots  (at  least  that  heading  mode  is  engaged.) 

•  This  is  a  very  nice  option.  I  like  being  able  to  give  steering  commands  to  the  LAR. 

•  Given  the  extra  SA  available  to  the  OSO,  this  function  could  provide  immediate 
access  to  aircraft  heading  control.  Could  improve  reaction  time  with  the  proper  use 
of  CRM,  i.e.  inform  pilot  if  the  OSO  is  changing  A/C  attitude.  (I  like  this  feature.) 

•  Good  heading  mode  capability,  but  a  critical  flight  coordination  item!  Has  to  be  a 
way  for  pilot  to  disengage  the  heading  mode. 

•  Even  with  heading  mode  engaged  it  would  be  convenient  for  the  system  to  sequence 

with  all  weapons  out  and  past  target. _ 
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4.  Rate  the  operational  utility  of  the  LAR  display  for  lining  up  multiple  LARs.  Use  the 


following  scale: 

Ratings: 

7  Totally  Acceptable  2 

6  Very  Acceptable  7 

5  Somewhat  Acceptable  1 

4  Borderline  0 

3  Somewhat  Unacceptable  0 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  6.10 


Comments: 

•  Very  useful,  especially  for  vertically  constrained  targets. 

•  Good/excellent  display  for  multiple  release,  yet  due  to  MFD  only  8  are  displayed 
whenever  your  target  is  8  or  more  JDAM. 

•  Overall  superb! 

•  Much  better  than  what  is  available  in  Block  D. 

•  Other  than  time  to  LAR  -  a  graphic  presentation  for  LAR  separation  (TTG)  would  be 
good. 

•  In  the  event  of  multiple  LARs,  the  pylon  function  rapidly  decreases  in  utility.  This 
LAR  display  becomes  very  important  for  SA  construction. 

•  Much  better  than  Block-D  displays  especially  with  arrows  with  a  heading  to  come  to. 

Once  in  near  ranging  especially  in  middle  of  re-allocation  or  off  track  for 
maneuvering  can  be  difficult. _ 
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5.  Were  you  able  to  follow  the  sequence  of  weapons  releases  using  the  LAR  Display? 
Yes  10  No  0  Comments: 


•  WSO  needs  to  get  used  to  bottom-up  display. 

•  Very  Good! 

•  Bottom-to-top  is  good  on  sequence  of  release. 

•  However,  I  had  a  tendency  to  start  at  the  top  of  the  [EB]  versus  the  bottom. 

•  But,  very  difficult  to  follow  during  a  multiple  ripple. 

•  Fairly  well. 

•  Great  job  on  time-to-exit  LAR _ 
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6.  Rate  the  acceptability  of  the  Inter-Crew  Coordination.  Use  the  following  scale: 


Ratings: 

7  Totally  Acceptable  0 

6  Very  Acceptable  3 

5  Somewhat  Acceptable  7 

4  Borderline  0 

3  Somewhat  Unacceptable  0 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  5.30 


Comments: 


•  With  practice  it  would  improve. 

•  More  comm  was  needed  to  maneuver  AC  to  LAR.  Having  not  flown  Block  D,  no 
technique  for  MAN/Auto  launch  and  platform  navigation  and  time  control  seems  to 
be  deferred  to  DSO  with  weapons  faults. 

•  Due  to  ERS  limitation  between  pilots  and  WSOs  it  worked  ok! 

•  Procedures  will  be  developed  (techniques  as  well). 

•  Pilots  still  [need]  their  own  LAR  SA. 

•  Remember  heading  mode  disengagement  is  critical  for  crew  coordination  in  threat 
area. 
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7.  Rate  the  acceptability  of  the  overall  Target  Situational  Awareness.  Use  the  following 
scale: 


Ratings: 

7  Totally  Acceptable  2 

6  Very  Acceptable  5 

5  Somewhat  Acceptable  3 

4  Borderline  0 

3  Somewhat  Unacceptable  0 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  5.90 


Comments: 


•  For  "black-line"  runs  [EB]  page  was  helpful. 

•  SA  can  be  lost  reallocating  weapons  if  system  doesn't  do  it.  The  map  (pylon  sw)  does 
wonders  after  getting  distracted  updating  weapons.  A  few  seconds  and  SA  is  back. 

•  Gave  us  the  "big  picture." 

•  Ability  to  select  different  ranges  on  map  enable. 

•  Map  enable  is  a  very  good  tool  for  helping  out  in  target  SA  -  would  like  to  have 
different  ranges  available. 

•  Map  enable  is  a  must.  It  was  nice  to  see  the  dynamic  bi-ellipses. 

•  Somewhat  confusing  when  re-allocations  occur. 

•  Once  you  get  used  to  it,  actually  gives  great  information! 

•  Bullseye  information  great!  Also  altitude  readout  is  good. 

•  Using  the  pylon  switch  to  bring  up  the  map  increases  target  SA. _ 
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8.  Rate  the  acceptability  of  the  Target  Situational  Awareness  with  respect  to  deviations 
from  the  planned  route.  Use  the  following  scale: 

Ratings: 


7  Totally  Acceptable  1 

6  Very  Acceptable  6 

5  Somewhat  Acceptable  2 

4  Borderline  1 


3  Somewhat  Unacceptable 
2  Very  Unacceptable 
1  Totally  Unacceptable 

RATING: _  Average:  5.70 

Comments: 


•  The  [E]  page  provides  more  SA  for  me  since  there  is  a  degrees  L/R  display.  Maybe 
with  practice,  the  [EB]  page  will  be  feasible. 

•  Map  is  great  with  LARs.  Understanding  LARs  on  EB  page  and  how  turning  affects 
page  is  a  must. 

•  Overall  super  -  especially  with  map  enable. 

•  Map  enable  really  helps  with  deviation.  Without  it,  it  would  be  more  difficult  to  keep 
S.A. 

•  This  is  acceptable  because  of  map  enable  feature. 

•  Once  again  the  pylon  mode  was  useful. 

•  LAR  displays  with  deviations  is  hard  when  not  in  Near  mode  ranging  especially  if 
close  to  DMPIs. 

•  Moving  LAR  on  map  is  a  little  confusing  at  first.  Would  not  be  a  problem  with 

practice _ 
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9.  Rate  the  implementation  of  the  Track  Handle.  Use  the  following  scale: 


Ratings: 

7  Totally  Acceptable  3 

6  Very  Acceptable  4 

5  Somewhat  Acceptable  3 

4  Borderline  0 

3  Somewhat  Unacceptable  0 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  6.00 


Comments: 


•  During  threat  reactions,  or  moving  from  one  page  to  another  de-selecting  steering 
command  is  easy  to  drop  out  of  crosscheck. 

•  Pilots  need  indication  of  what  heading  mode  OSO  is  in. 

•  We  reviewed  and  worked  the  "LAR"  option  of  the  track  handle.  The  ILST  with  track 
handle  options  may  add  more  stress/workload  for  WSOs. 

•  Track  handle  function  is  great.  Add. 

•  A  more  visible  reminder  when  the  track  handle  has  heading. _ 
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10.  Rate  the  operational  utility  of  the  Declutter  Mode.  Use  the  following  scale: 


Ratings: 

7  Totally  Acceptable  3 

6  Very  Acceptable  5 

5  Somewhat  Acceptable  1 

4  Borderline  1 

3  Somewhat  Unacceptable  0 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  6.00 


Comments: 


•  If  4  or  less  targets,  much  easier  to  read  screen. 

•  Superb!!  Keep  it! 

•  Good  in  that  it  lets  you  concentrate  on  just  4  of  the  releases  (if  desired). 

•  Only  necessary  when  accomplishing  a  multiple  release  of  10  or  more. 

•  Did  not  have  a  chance  to  try  this. 

•  Declutter  helps  to  not  get  too  much  info. _ 
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11.  Rate  the  acceptability  of  the  scale  modes.  Use  the  following  scales: 


a) .  60° 

7  Totally  Acceptable 

6  Very  Acceptable 

5  Somewhat  Acceptable 

4  Borderline 

3  Somewhat  Unacceptable 

2  Very  Unacceptable 

1  Totally  Unacceptable 

b) .  30° 

7  Totally  Acceptable 

6  Very  Acceptable 

5  Somewhat  Acceptable 

4  Borderline 

3  Somewhat  Unacceptable 

2  Very  Unacceptable 

1  Totally  Unacceptable 

c) .  10° 

7  Totally  Acceptable 

6  Very  Acceptable 

5  Somewhat  Acceptable 

4  Borderline 

3  Somewhat  Unacceptable 

2  Very  Unacceptable 

1  Totally  Unacceptable 


Ratings: 

4 

3 

3 
0 
0 
0 
0 

Average:  6.10 

Ratings: 

4 
3 

3 
0 
0 
0 
0 

Average:  6.10 

Ratings: 

4 
1 
4 
1 
0 
0 
0 

Average:  5.80 


Comments: 

•  I  did  not  use  the  30°  or  10°  due  to  task  saturation  and  limits  on  range  size. 

•  Keep  it! ! 

•  Did  not  use  this.  Seems  conceptually  useful. 

•  Did  not  utilize  this  function  much. 
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12.  Rate  the  operational  utility  of  the  Time-to-go/Time-in-LAR  information  on  the  EB 
page.  Use  the  following  scale: 


Ratings: 

7  Totally  Acceptable  4 

6  Very  Acceptable  6 

5  Somewhat  Acceptable  0 

4  Borderline  0 

3  Somewhat  Unacceptable  0 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  6.40 


Comments: 


•  TTG/Time  in  LAR  used  in  combination  with  map  is  good. 

•  Keep  it!  After  WSO  gets  used  to  the  formats  -  information  offers  same  data  as  Block 
D  [I]  page. 

•  Very  useful  information  except  the  Time-In-LAR  information  was  based  on  "best 
case"  intercept  across  farthest  points.  If  you  are  just  inside  the  left  edge  of  the  LAR 
you  will  transit  the  LAR  much  quicker. 

•  I  like  it! 
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13.  On  the  EB  page,  which  drop  order  do  you  prefer? 
Top-to-bottom  3 
Bottom-to-top  7_ 

Comments: 


•  Top-to-Bottom  doesn't  make  any  sense. 

•  It's  fine/OK 

•  As  you  fly  into  the  LAR! _ 
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14.  Rate  the  operational  utility  of  the  angular  representation  on  the  LAR  bar.  Use  the 
following  scale: 

Ratings: 

7  Totally  Acceptable 

3 

6  Very  Acceptable 

2 

5  Somewhat  Acceptable 

5 

4  Borderline 

0 

3  Somewhat  Unacceptable 

0 

2  Very  Unacceptable 

0 

1  Totally  Unacceptable 

0 

RATING: 

Comments: 

Average:  5.80 

•  Represents  in  relation  to  distance.  Keep  it! 

•  OK,  but  when  in  Near  ranging  with  no  azimuth  constraint  I  am  looking  to  put  or  line 
up  (center)  the  A/C  in  the  heart  of  the  LAR. 
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15.  Rate  the  operational  utility  of  the  Magnetic  Heading  information  on  the  EB  page.  Use 
the  following  scale: 

Ratings: 


7  Totally  Acceptable  3 

6  Very  Acceptable  5 

5  Somewhat  Acceptable  2 

4  Borderline  0 

3  Somewhat  Unacceptable  0 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  6.10 


Comments: 


Nice,  but  most  people  use  HSI. 

Keep  it  /  Superb! 

Old  dog  -  still  need  to  break  away  from  GNAV  summary. 

During  HDG  MODE,  heading  useful. 

When  in  HDG  mode  need  to  know  what  heading  A/C  is  presently  on. 
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16.  Rate  the  operational  utility  of  the  Digital  Bull  and  Magnetic  Course  information  on 
the  Nav  Prime  Display.  Use  the  following  scale: 

Ratings: 


7  Totally  Acceptable  9 

6  Very  Acceptable  1 

5  Somewhat  Acceptable  0 

4  Borderline  0 

3  Somewhat  Unacceptable  0 

2  Very  Unacceptable  0 

1  Totally  Unacceptable  0 

RATING: _  Average:  6.90 


Comments: 


•  Digital  Bull  in  the  B-l  a  must! 

•  Bull  is  a  must.  Multiple  Bulls  and  ability  to  change  it  is  needed  also. 

•  Definitely  keep  it! 

•  This  provides  massive  amounts  of  SA. 

•  We  need  this  feature  now! ! 

•  This  is  huge.  The  B-1B  has  a  hard  time  maintaining  SA  during  an  air-to-air  merge. 
This  information  is  a  source  of  instant  Bull  location  of  A/C.  The  Bull  information  on 
the  DSO  scope  is  also  huge.  Wish  we  had  it  coming  too.  Gives  us  threat  bull  info  as 
well.. 

•  Both  great! _ 
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General  Comments: 


•  Without  Block  D  experience. 

•  Eye  opening  from  perspective  of  having  heard  some  of  office  critiques  of  Block  D. 
ALT  LAR  is  straightforward  with  experience  and  understanding  of  how  turns  effect 
LAR.  Graphical  map  does  wonders  for  SA.  Block  F  DAS  is  nice  with  compass  rose 
and  bull. 
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APPENDIX  2.  STATISTICS  TABLES 


Table  4.  Correlations  Among  Variables 


freeze  pt 

SWAT 

EBPCT 

MAPPCT 

freeze  pt 

Pearson  Correlation 

1.000 

.009 

.035 

-.354 

Sig.  (2-tailed) 

.962 

.852 

.055 

N 

30 

30 

30 

30 

SWAT 

Pearson  Correlation 

.009 

1.000 

-.062 

.069 

Sig.  (2-tailed) 

.962 

.744 

.719 

N 

30 

30 

30 

30 

EBPCT 

Pearson  Correlation 

.035 

-.062 

1.000 

.295 

Sig.  (2-tailed) 

.852 

.744 

.114 

N 

30 

30 

30 

30 

MAPPCT 

Pearson  Correlation 

-.354 

.069 

.295 

1.000 

Sig.  (2-tailed) 

.055 

.719 

.114 

N 

30 

30 

30 

30 

Table  5.  Multivariate  Effect  of  Freeze  Point  on  SWAT 


Effect 

Value 

F 

Hypothesis  df 

Error  df 

Sig. 

Freeze 

Point 

Pillai's  Trace 

.385 

2.507(a) 

2.000 

8.000 

.143 

Wilks'  Lambda 

.615 

2.507(a) 

2.000 

8.000 

.143 

Hotelling's  Trace 

.627 

2.507(a) 

2.000 

8.000 

.143 

Roy's  Largest  Root 

.627 

2.507(a) 

2.000 

8.000 

.143 

a  Exact  statistic 


b  Design:  Intercept 

Within  Subjects  Design:  FRZ_PT 
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Table  6.  Multivariate  Effect  of  Freeze  Point 
and  Experience  on  SWAT 


Effect 

Value 

F 

Hypothesis 

df 

Error  df 

Sig. 

Freeze  Point 

Pillai's  Trace 

.476 

2.273(a) 

2.000 

5.000 

.199 

Wilks'  Lambda 

.524 

2.273(a) 

2.000 

5.000 

.199 

Hotelling's 

Trace 

.909 

2.273(a) 

2.000 

5.000 

.199 

Roy's  Largest 
Root 

.909 

2.273(a) 

2.000 

5.000 

.199 

FRZ  PT* 
EXPERBIN 

Pillai's  Trace 

.268 

.309 

6.000 

12.000 

.920 

Wilks'  Lambda 

.738 

.274(a) 

6.000 

10.000 

.937 

Hotelling's 

Trace 

.347 

.232 

6.000 

8.000 

.954 

Roy's  Largest 
Root 

.323 

.646(b) 

3.000 

6.000 

.613 

a  Exact  statistic 

b  The  statistic  is  an  upper  bound  on  F  that  yields  a  lower  bound  on  the  significance  level. 

c  Design:  Intercept+EXPERBIN 

Within  Subjects  Design:  FRZ_PT 

Table  7.  Multivariate  Effect  of  Freeze  Point  with  Experience  on  EB  Page  Percent  of  Time 


Effect 

Value 

F 

Hypothesis 

df 

Error  df 

Sig. 

Eta 

Squared 

FRZ_PT 

Pillai's  T  race 

.411 

1.743(a) 

2.000 

5.000 

.267 

.411 

Wilks'  Lambda 

.589 

1.743(a) 

2.000 

5.000 

.267 

.411 

Hotelling's 

Trace 

.697 

1.743(a) 

2.000 

5.000 

.267 

.411 

Roy's  Largest 
Root 

.697 

1.743(a) 

2.000 

5.000 

.267 

.411 

FRZ  PT* 
EXPERBIN 

Pillai's  T  race 

1.197 

2.981 

6.000 

12.000 

.051 

.598 

Wilks'  Lambda 

.158 

2.524(a) 

6.000 

10.000 

.094 

.602 

Hotelling's 

Trace 

3.076 

2.050 

6.000 

8.000 

.171 

.606 

Roy's  Largest 
Root 

1.884 

3.768(b) 

3.000 

6.000 

.078 

.653 

a  Exact  statistic 

b  The  statistic  is  an  upper  bound  on  F  that  yields  a  lower  bound  on  the  significance  level. 

c  Design:  Intercept+EXPERBIN 

Within  Subjects  Design:  FRZ_PT 

Table  8.  Simultaneous  Comparison  of  Freeze  Point 
Means  for  Map  Enable 
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Mean  Difference  (1- 
J) 

Std. 

Error 

Sig.(a) 

95%  Confidence  Interval  for 
Difference(a) 

(1) 

FRZPT 

(J) 

FRZPT 

Lower  Bound 

Upper  Bound 

1 

2 

27.804 

11.225 

.144 

-9.098 

64.706 

3 

21.479 

14.802 

.591 

-27.183 

70.141 

2 

1 

-27.804 

11.225 

.144 

-64.706 

9.098 

3 

-6.325 

5.021 

.764 

-22.833 

10.183 

3 

1 

-21.479 

14.802 

.591 

-70.141 

27.183 

2 

6.325 

5.021 

.764 

-10.183 

22.833 

Based  on  estimated  marginal  means 


a  Adjustment  for  multiple  comparisons:  Bonferroni. 
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LIST  OF  ACRONYMS 


ALT  LAR 

Alternative  Launch  Acceptability  Region 

CMUP 

Conventional  Mission  Upgrade  Program 

CSEF 

Crew  Station  Evaluation  Facility 

CSWG 

Crew  Station  Working  Group 

DMPI 

Desired  Mean  Point  of  Impact 

DSO 

Defensive  System  Operator 

DSUP 

Defensive  System  Upgrade  Program 

E  page 

Target  Summary  Display 

EB  page 

LAR  Status  Display 

ERS 

Engineering  Research  Simulator 

GPS 

Global  Positioning  System 

INS 

Inertial  Navigation  System 

JDAM 

Joint  Direct  Attack  Munition 

LAR 

Launch  Acceptability  Region 

MFD 

Multi-Function  Display 

Nav  Prime 

Primary  Navigation 

oso 

Offensive  Systems  Operator 

RWR 

Radar  Warning  Receiver 

SPO 

System  Program  Office 

SWAT 

Subjective  Workload  Assessment  Technique 

TTG 

Time-To-Go 

WSO 

Weapons  Systems  Operators 
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